It is impractical to have a large size of training dataset in real work for neural network training, so a two-input stream generative neural network which can generate a new image with the given parameters was proposed, hence to augment the training dataset. The framework of the proposed neural network consists of a two-input steam convolution network and a deconvolution network. The two-input steam network has two convolution networks to extract features, and the deconvolution network is connected to the end. Two images with different angle were input into the convolution network to get high-level description, then an interpolation target image with a new perspectives was generated by using the deconvolution network with the above high-level description and set parameters. The experiment results on ShapeNetCore show that on the same dataset, the recognition rate of the proposed network has increased by 20% than the common network framework. This method can enlarge the size of the training dataset and is useful for multi-angle recognition.